3.1.55 \(\int \frac {\cot ^2(x)}{(a+b \cot ^2(x))^{5/2}} \, dx\) [55]

3.1.55.1 Optimal result
3.1.55.2 Mathematica [C] (warning: unable to verify)
3.1.55.3 Rubi [A] (verified)
3.1.55.4 Maple [A] (verified)
3.1.55.5 Fricas [B] (verification not implemented)
3.1.55.6 Sympy [F]
3.1.55.7 Maxima [F(-2)]
3.1.55.8 Giac [B] (verification not implemented)
3.1.55.9 Mupad [F(-1)]

3.1.55.1 Optimal result

Integrand size = 17, antiderivative size = 94 \[ \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx=\frac {\arctan \left (\frac {\sqrt {a-b} \cot (x)}{\sqrt {a+b \cot ^2(x)}}\right )}{(a-b)^{5/2}}-\frac {\cot (x)}{3 (a-b) \left (a+b \cot ^2(x)\right )^{3/2}}-\frac {(2 a+b) \cot (x)}{3 a (a-b)^2 \sqrt {a+b \cot ^2(x)}} \]

output
arctan(cot(x)*(a-b)^(1/2)/(a+b*cot(x)^2)^(1/2))/(a-b)^(5/2)-1/3*cot(x)/(a- 
b)/(a+b*cot(x)^2)^(3/2)-1/3*(2*a+b)*cot(x)/a/(a-b)^2/(a+b*cot(x)^2)^(1/2)
 
3.1.55.2 Mathematica [C] (warning: unable to verify)

Result contains higher order function than in optimal. Order 5 vs. order 3 in optimal.

Time = 6.34 (sec) , antiderivative size = 197, normalized size of antiderivative = 2.10 \[ \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx=\frac {\cos (x) \left (-12 (a-b)^3 \cos ^3(x) \cot (x) \left (a+b \cot ^2(x)\right ) \operatorname {Hypergeometric2F1}\left (2,2,\frac {9}{2},\frac {(a-b) \cos ^2(x)}{a}\right )-\frac {35 a \left (5 a+2 b \cot ^2(x)\right ) \sin (x) \left (a \left ((a-4 b) \csc ^2(x)-3 a \sec ^2(x)\right ) \sqrt {\frac {(a-b) \cos ^2(x) \left (a+b \cot ^2(x)\right ) \sin ^2(x)}{a^2}}+3 \arcsin \left (\sqrt {\frac {(a-b) \cos ^2(x)}{a}}\right ) (b \cot (x)+a \tan (x))^2\right )}{\sqrt {\frac {(a-b) \cos ^2(x) \left (a+b \cot ^2(x)\right ) \sin ^2(x)}{a^2}}}\right )}{315 a^3 (a-b)^2 \left (a+b \cot ^2(x)\right )^{3/2}} \]

input
Integrate[Cot[x]^2/(a + b*Cot[x]^2)^(5/2),x]
 
output
(Cos[x]*(-12*(a - b)^3*Cos[x]^3*Cot[x]*(a + b*Cot[x]^2)*Hypergeometric2F1[ 
2, 2, 9/2, ((a - b)*Cos[x]^2)/a] - (35*a*(5*a + 2*b*Cot[x]^2)*Sin[x]*(a*(( 
a - 4*b)*Csc[x]^2 - 3*a*Sec[x]^2)*Sqrt[((a - b)*Cos[x]^2*(a + b*Cot[x]^2)* 
Sin[x]^2)/a^2] + 3*ArcSin[Sqrt[((a - b)*Cos[x]^2)/a]]*(b*Cot[x] + a*Tan[x] 
)^2))/Sqrt[((a - b)*Cos[x]^2*(a + b*Cot[x]^2)*Sin[x]^2)/a^2]))/(315*a^3*(a 
 - b)^2*(a + b*Cot[x]^2)^(3/2))
 
3.1.55.3 Rubi [A] (verified)

Time = 0.31 (sec) , antiderivative size = 105, normalized size of antiderivative = 1.12, number of steps used = 8, number of rules used = 7, \(\frac {\text {number of rules}}{\text {integrand size}}\) = 0.412, Rules used = {3042, 4153, 373, 402, 27, 291, 216}

Below are the steps used by Rubi to obtain the solution. The rule number used for the transformation is given above next to the arrow. The rules definitions used are listed below.

\(\displaystyle \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx\)

\(\Big \downarrow \) 3042

\(\displaystyle \int \frac {\tan \left (x+\frac {\pi }{2}\right )^2}{\left (a+b \tan \left (x+\frac {\pi }{2}\right )^2\right )^{5/2}}dx\)

\(\Big \downarrow \) 4153

\(\displaystyle -\int \frac {\cot ^2(x)}{\left (\cot ^2(x)+1\right ) \left (b \cot ^2(x)+a\right )^{5/2}}d\cot (x)\)

\(\Big \downarrow \) 373

\(\displaystyle \frac {\int \frac {1-2 \cot ^2(x)}{\left (\cot ^2(x)+1\right ) \left (b \cot ^2(x)+a\right )^{3/2}}d\cot (x)}{3 (a-b)}-\frac {\cot (x)}{3 (a-b) \left (a+b \cot ^2(x)\right )^{3/2}}\)

\(\Big \downarrow \) 402

\(\displaystyle \frac {\frac {\int \frac {3 a}{\left (\cot ^2(x)+1\right ) \sqrt {b \cot ^2(x)+a}}d\cot (x)}{a (a-b)}-\frac {(2 a+b) \cot (x)}{a (a-b) \sqrt {a+b \cot ^2(x)}}}{3 (a-b)}-\frac {\cot (x)}{3 (a-b) \left (a+b \cot ^2(x)\right )^{3/2}}\)

\(\Big \downarrow \) 27

\(\displaystyle \frac {\frac {3 \int \frac {1}{\left (\cot ^2(x)+1\right ) \sqrt {b \cot ^2(x)+a}}d\cot (x)}{a-b}-\frac {(2 a+b) \cot (x)}{a (a-b) \sqrt {a+b \cot ^2(x)}}}{3 (a-b)}-\frac {\cot (x)}{3 (a-b) \left (a+b \cot ^2(x)\right )^{3/2}}\)

\(\Big \downarrow \) 291

\(\displaystyle \frac {\frac {3 \int \frac {1}{1-\frac {(b-a) \cot ^2(x)}{b \cot ^2(x)+a}}d\frac {\cot (x)}{\sqrt {b \cot ^2(x)+a}}}{a-b}-\frac {(2 a+b) \cot (x)}{a (a-b) \sqrt {a+b \cot ^2(x)}}}{3 (a-b)}-\frac {\cot (x)}{3 (a-b) \left (a+b \cot ^2(x)\right )^{3/2}}\)

\(\Big \downarrow \) 216

\(\displaystyle \frac {\frac {3 \arctan \left (\frac {\sqrt {a-b} \cot (x)}{\sqrt {a+b \cot ^2(x)}}\right )}{(a-b)^{3/2}}-\frac {(2 a+b) \cot (x)}{a (a-b) \sqrt {a+b \cot ^2(x)}}}{3 (a-b)}-\frac {\cot (x)}{3 (a-b) \left (a+b \cot ^2(x)\right )^{3/2}}\)

input
Int[Cot[x]^2/(a + b*Cot[x]^2)^(5/2),x]
 
output
-1/3*Cot[x]/((a - b)*(a + b*Cot[x]^2)^(3/2)) + ((3*ArcTan[(Sqrt[a - b]*Cot 
[x])/Sqrt[a + b*Cot[x]^2]])/(a - b)^(3/2) - ((2*a + b)*Cot[x])/(a*(a - b)* 
Sqrt[a + b*Cot[x]^2]))/(3*(a - b))
 

3.1.55.3.1 Defintions of rubi rules used

rule 27
Int[(a_)*(Fx_), x_Symbol] :> Simp[a   Int[Fx, x], x] /; FreeQ[a, x] &&  !Ma 
tchQ[Fx, (b_)*(Gx_) /; FreeQ[b, x]]
 

rule 216
Int[((a_) + (b_.)*(x_)^2)^(-1), x_Symbol] :> Simp[(1/(Rt[a, 2]*Rt[b, 2]))*A 
rcTan[Rt[b, 2]*(x/Rt[a, 2])], x] /; FreeQ[{a, b}, x] && PosQ[a/b] && (GtQ[a 
, 0] || GtQ[b, 0])
 

rule 291
Int[1/(Sqrt[(a_) + (b_.)*(x_)^2]*((c_) + (d_.)*(x_)^2)), x_Symbol] :> Subst 
[Int[1/(c - (b*c - a*d)*x^2), x], x, x/Sqrt[a + b*x^2]] /; FreeQ[{a, b, c, 
d}, x] && NeQ[b*c - a*d, 0]
 

rule 373
Int[((e_.)*(x_))^(m_.)*((a_) + (b_.)*(x_)^2)^(p_)*((c_) + (d_.)*(x_)^2)^(q_ 
), x_Symbol] :> Simp[e*(e*x)^(m - 1)*(a + b*x^2)^(p + 1)*((c + d*x^2)^(q + 
1)/(2*(b*c - a*d)*(p + 1))), x] - Simp[e^2/(2*(b*c - a*d)*(p + 1))   Int[(e 
*x)^(m - 2)*(a + b*x^2)^(p + 1)*(c + d*x^2)^q*Simp[c*(m - 1) + d*(m + 2*p + 
 2*q + 3)*x^2, x], x], x] /; FreeQ[{a, b, c, d, e, q}, x] && NeQ[b*c - a*d, 
 0] && LtQ[p, -1] && GtQ[m, 1] && LeQ[m, 3] && IntBinomialQ[a, b, c, d, e, 
m, 2, p, q, x]
 

rule 402
Int[((a_) + (b_.)*(x_)^2)^(p_)*((c_) + (d_.)*(x_)^2)^(q_.)*((e_) + (f_.)*(x 
_)^2), x_Symbol] :> Simp[(-(b*e - a*f))*x*(a + b*x^2)^(p + 1)*((c + d*x^2)^ 
(q + 1)/(a*2*(b*c - a*d)*(p + 1))), x] + Simp[1/(a*2*(b*c - a*d)*(p + 1)) 
 Int[(a + b*x^2)^(p + 1)*(c + d*x^2)^q*Simp[c*(b*e - a*f) + e*2*(b*c - a*d) 
*(p + 1) + d*(b*e - a*f)*(2*(p + q + 2) + 1)*x^2, x], x], x] /; FreeQ[{a, b 
, c, d, e, f, q}, x] && LtQ[p, -1]
 

rule 3042
Int[u_, x_Symbol] :> Int[DeactivateTrig[u, x], x] /; FunctionOfTrigOfLinear 
Q[u, x]
 

rule 4153
Int[((d_.)*tan[(e_.) + (f_.)*(x_)])^(m_.)*((a_) + (b_.)*((c_.)*tan[(e_.) + 
(f_.)*(x_)])^(n_))^(p_.), x_Symbol] :> With[{ff = FreeFactors[Tan[e + f*x], 
 x]}, Simp[c*(ff/f)   Subst[Int[(d*ff*(x/c))^m*((a + b*(ff*x)^n)^p/(c^2 + f 
f^2*x^2)), x], x, c*(Tan[e + f*x]/ff)], x]] /; FreeQ[{a, b, c, d, e, f, m, 
n, p}, x] && (IGtQ[p, 0] || EqQ[n, 2] || EqQ[n, 4] || (IntegerQ[p] && Ratio 
nalQ[n]))
 
3.1.55.4 Maple [A] (verified)

Time = 0.06 (sec) , antiderivative size = 161, normalized size of antiderivative = 1.71

method result size
derivativedivides \(-\frac {\cot \left (x \right )}{3 a \left (a +b \cot \left (x \right )^{2}\right )^{\frac {3}{2}}}-\frac {2 \cot \left (x \right )}{3 a^{2} \sqrt {a +b \cot \left (x \right )^{2}}}-\frac {b \left (\frac {\cot \left (x \right )}{3 a \left (a +b \cot \left (x \right )^{2}\right )^{\frac {3}{2}}}+\frac {2 \cot \left (x \right )}{3 a^{2} \sqrt {a +b \cot \left (x \right )^{2}}}\right )}{a -b}-\frac {b \cot \left (x \right )}{\left (a -b \right )^{2} a \sqrt {a +b \cot \left (x \right )^{2}}}+\frac {\sqrt {b^{4} \left (a -b \right )}\, \arctan \left (\frac {b^{2} \left (a -b \right ) \cot \left (x \right )}{\sqrt {b^{4} \left (a -b \right )}\, \sqrt {a +b \cot \left (x \right )^{2}}}\right )}{\left (a -b \right )^{3} b^{2}}\) \(161\)
default \(-\frac {\cot \left (x \right )}{3 a \left (a +b \cot \left (x \right )^{2}\right )^{\frac {3}{2}}}-\frac {2 \cot \left (x \right )}{3 a^{2} \sqrt {a +b \cot \left (x \right )^{2}}}-\frac {b \left (\frac {\cot \left (x \right )}{3 a \left (a +b \cot \left (x \right )^{2}\right )^{\frac {3}{2}}}+\frac {2 \cot \left (x \right )}{3 a^{2} \sqrt {a +b \cot \left (x \right )^{2}}}\right )}{a -b}-\frac {b \cot \left (x \right )}{\left (a -b \right )^{2} a \sqrt {a +b \cot \left (x \right )^{2}}}+\frac {\sqrt {b^{4} \left (a -b \right )}\, \arctan \left (\frac {b^{2} \left (a -b \right ) \cot \left (x \right )}{\sqrt {b^{4} \left (a -b \right )}\, \sqrt {a +b \cot \left (x \right )^{2}}}\right )}{\left (a -b \right )^{3} b^{2}}\) \(161\)

input
int(cot(x)^2/(a+b*cot(x)^2)^(5/2),x,method=_RETURNVERBOSE)
 
output
-1/3*cot(x)/a/(a+b*cot(x)^2)^(3/2)-2/3/a^2*cot(x)/(a+b*cot(x)^2)^(1/2)-1/( 
a-b)*b*(1/3*cot(x)/a/(a+b*cot(x)^2)^(3/2)+2/3/a^2*cot(x)/(a+b*cot(x)^2)^(1 
/2))-1/(a-b)^2*b*cot(x)/a/(a+b*cot(x)^2)^(1/2)+1/(a-b)^3*(b^4*(a-b))^(1/2) 
/b^2*arctan(b^2*(a-b)/(b^4*(a-b))^(1/2)/(a+b*cot(x)^2)^(1/2)*cot(x))
 
3.1.55.5 Fricas [B] (verification not implemented)

Leaf count of result is larger than twice the leaf count of optimal. 339 vs. \(2 (80) = 160\).

Time = 0.39 (sec) , antiderivative size = 720, normalized size of antiderivative = 7.66 \[ \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx=\left [-\frac {3 \, {\left (a^{3} + 2 \, a^{2} b + a b^{2} + {\left (a^{3} - 2 \, a^{2} b + a b^{2}\right )} \cos \left (2 \, x\right )^{2} - 2 \, {\left (a^{3} - a b^{2}\right )} \cos \left (2 \, x\right )\right )} \sqrt {-a + b} \log \left (-2 \, {\left (a^{2} - 2 \, a b + b^{2}\right )} \cos \left (2 \, x\right )^{2} + 2 \, {\left ({\left (a - b\right )} \cos \left (2 \, x\right ) - b\right )} \sqrt {-a + b} \sqrt {\frac {{\left (a - b\right )} \cos \left (2 \, x\right ) - a - b}{\cos \left (2 \, x\right ) - 1}} \sin \left (2 \, x\right ) + a^{2} - 2 \, b^{2} + 4 \, {\left (a b - b^{2}\right )} \cos \left (2 \, x\right )\right ) + 4 \, {\left (3 \, a^{3} - a^{2} b - a b^{2} - b^{3} - {\left (3 \, a^{3} - 5 \, a^{2} b + a b^{2} + b^{3}\right )} \cos \left (2 \, x\right )\right )} \sqrt {\frac {{\left (a - b\right )} \cos \left (2 \, x\right ) - a - b}{\cos \left (2 \, x\right ) - 1}} \sin \left (2 \, x\right )}{12 \, {\left (a^{6} - a^{5} b - 2 \, a^{4} b^{2} + 2 \, a^{3} b^{3} + a^{2} b^{4} - a b^{5} + {\left (a^{6} - 5 \, a^{5} b + 10 \, a^{4} b^{2} - 10 \, a^{3} b^{3} + 5 \, a^{2} b^{4} - a b^{5}\right )} \cos \left (2 \, x\right )^{2} - 2 \, {\left (a^{6} - 3 \, a^{5} b + 2 \, a^{4} b^{2} + 2 \, a^{3} b^{3} - 3 \, a^{2} b^{4} + a b^{5}\right )} \cos \left (2 \, x\right )\right )}}, \frac {3 \, {\left (a^{3} + 2 \, a^{2} b + a b^{2} + {\left (a^{3} - 2 \, a^{2} b + a b^{2}\right )} \cos \left (2 \, x\right )^{2} - 2 \, {\left (a^{3} - a b^{2}\right )} \cos \left (2 \, x\right )\right )} \sqrt {a - b} \arctan \left (-\frac {\sqrt {a - b} \sqrt {\frac {{\left (a - b\right )} \cos \left (2 \, x\right ) - a - b}{\cos \left (2 \, x\right ) - 1}} \sin \left (2 \, x\right )}{{\left (a - b\right )} \cos \left (2 \, x\right ) - b}\right ) - 2 \, {\left (3 \, a^{3} - a^{2} b - a b^{2} - b^{3} - {\left (3 \, a^{3} - 5 \, a^{2} b + a b^{2} + b^{3}\right )} \cos \left (2 \, x\right )\right )} \sqrt {\frac {{\left (a - b\right )} \cos \left (2 \, x\right ) - a - b}{\cos \left (2 \, x\right ) - 1}} \sin \left (2 \, x\right )}{6 \, {\left (a^{6} - a^{5} b - 2 \, a^{4} b^{2} + 2 \, a^{3} b^{3} + a^{2} b^{4} - a b^{5} + {\left (a^{6} - 5 \, a^{5} b + 10 \, a^{4} b^{2} - 10 \, a^{3} b^{3} + 5 \, a^{2} b^{4} - a b^{5}\right )} \cos \left (2 \, x\right )^{2} - 2 \, {\left (a^{6} - 3 \, a^{5} b + 2 \, a^{4} b^{2} + 2 \, a^{3} b^{3} - 3 \, a^{2} b^{4} + a b^{5}\right )} \cos \left (2 \, x\right )\right )}}\right ] \]

input
integrate(cot(x)^2/(a+b*cot(x)^2)^(5/2),x, algorithm="fricas")
 
output
[-1/12*(3*(a^3 + 2*a^2*b + a*b^2 + (a^3 - 2*a^2*b + a*b^2)*cos(2*x)^2 - 2* 
(a^3 - a*b^2)*cos(2*x))*sqrt(-a + b)*log(-2*(a^2 - 2*a*b + b^2)*cos(2*x)^2 
 + 2*((a - b)*cos(2*x) - b)*sqrt(-a + b)*sqrt(((a - b)*cos(2*x) - a - b)/( 
cos(2*x) - 1))*sin(2*x) + a^2 - 2*b^2 + 4*(a*b - b^2)*cos(2*x)) + 4*(3*a^3 
 - a^2*b - a*b^2 - b^3 - (3*a^3 - 5*a^2*b + a*b^2 + b^3)*cos(2*x))*sqrt((( 
a - b)*cos(2*x) - a - b)/(cos(2*x) - 1))*sin(2*x))/(a^6 - a^5*b - 2*a^4*b^ 
2 + 2*a^3*b^3 + a^2*b^4 - a*b^5 + (a^6 - 5*a^5*b + 10*a^4*b^2 - 10*a^3*b^3 
 + 5*a^2*b^4 - a*b^5)*cos(2*x)^2 - 2*(a^6 - 3*a^5*b + 2*a^4*b^2 + 2*a^3*b^ 
3 - 3*a^2*b^4 + a*b^5)*cos(2*x)), 1/6*(3*(a^3 + 2*a^2*b + a*b^2 + (a^3 - 2 
*a^2*b + a*b^2)*cos(2*x)^2 - 2*(a^3 - a*b^2)*cos(2*x))*sqrt(a - b)*arctan( 
-sqrt(a - b)*sqrt(((a - b)*cos(2*x) - a - b)/(cos(2*x) - 1))*sin(2*x)/((a 
- b)*cos(2*x) - b)) - 2*(3*a^3 - a^2*b - a*b^2 - b^3 - (3*a^3 - 5*a^2*b + 
a*b^2 + b^3)*cos(2*x))*sqrt(((a - b)*cos(2*x) - a - b)/(cos(2*x) - 1))*sin 
(2*x))/(a^6 - a^5*b - 2*a^4*b^2 + 2*a^3*b^3 + a^2*b^4 - a*b^5 + (a^6 - 5*a 
^5*b + 10*a^4*b^2 - 10*a^3*b^3 + 5*a^2*b^4 - a*b^5)*cos(2*x)^2 - 2*(a^6 - 
3*a^5*b + 2*a^4*b^2 + 2*a^3*b^3 - 3*a^2*b^4 + a*b^5)*cos(2*x))]
 
3.1.55.6 Sympy [F]

\[ \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx=\int \frac {\cot ^{2}{\left (x \right )}}{\left (a + b \cot ^{2}{\left (x \right )}\right )^{\frac {5}{2}}}\, dx \]

input
integrate(cot(x)**2/(a+b*cot(x)**2)**(5/2),x)
 
output
Integral(cot(x)**2/(a + b*cot(x)**2)**(5/2), x)
 
3.1.55.7 Maxima [F(-2)]

Exception generated. \[ \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx=\text {Exception raised: ValueError} \]

input
integrate(cot(x)^2/(a+b*cot(x)^2)^(5/2),x, algorithm="maxima")
 
output
Exception raised: ValueError >> Computation failed since Maxima requested 
additional constraints; using the 'assume' command before evaluation *may* 
 help (example of legal syntax is 'assume(b-a>0)', see `assume?` for more 
details)Is
 
3.1.55.8 Giac [B] (verification not implemented)

Leaf count of result is larger than twice the leaf count of optimal. 281 vs. \(2 (80) = 160\).

Time = 0.34 (sec) , antiderivative size = 281, normalized size of antiderivative = 2.99 \[ \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx=\frac {{\left (3 \, a \sqrt {b} \log \left ({\left | -\sqrt {-a + b} + \sqrt {b} \right |}\right ) + 2 \, a \sqrt {-a + b} + \sqrt {-a + b} b\right )} \mathrm {sgn}\left (\sin \left (x\right )\right )}{3 \, {\left (a^{3} \sqrt {-a + b} \sqrt {b} - 2 \, a^{2} \sqrt {-a + b} b^{\frac {3}{2}} + a \sqrt {-a + b} b^{\frac {5}{2}}\right )}} - \frac {\frac {{\left (\frac {{\left (3 \, a^{3} - 5 \, a^{2} b + a b^{2} + b^{3}\right )} \cos \left (x\right )^{2}}{a^{4} - 3 \, a^{3} b + 3 \, a^{2} b^{2} - a b^{3}} - \frac {3 \, {\left (a^{3} - a^{2} b\right )}}{a^{4} - 3 \, a^{3} b + 3 \, a^{2} b^{2} - a b^{3}}\right )} \cos \left (x\right )}{{\left (a \cos \left (x\right )^{2} - b \cos \left (x\right )^{2} - a\right )} \sqrt {-a \cos \left (x\right )^{2} + b \cos \left (x\right )^{2} + a}} + \frac {3 \, \log \left ({\left | -\sqrt {-a + b} \cos \left (x\right ) + \sqrt {-a \cos \left (x\right )^{2} + b \cos \left (x\right )^{2} + a} \right |}\right )}{{\left (a^{2} - 2 \, a b + b^{2}\right )} \sqrt {-a + b}}}{3 \, \mathrm {sgn}\left (\sin \left (x\right )\right )} \]

input
integrate(cot(x)^2/(a+b*cot(x)^2)^(5/2),x, algorithm="giac")
 
output
1/3*(3*a*sqrt(b)*log(abs(-sqrt(-a + b) + sqrt(b))) + 2*a*sqrt(-a + b) + sq 
rt(-a + b)*b)*sgn(sin(x))/(a^3*sqrt(-a + b)*sqrt(b) - 2*a^2*sqrt(-a + b)*b 
^(3/2) + a*sqrt(-a + b)*b^(5/2)) - 1/3*(((3*a^3 - 5*a^2*b + a*b^2 + b^3)*c 
os(x)^2/(a^4 - 3*a^3*b + 3*a^2*b^2 - a*b^3) - 3*(a^3 - a^2*b)/(a^4 - 3*a^3 
*b + 3*a^2*b^2 - a*b^3))*cos(x)/((a*cos(x)^2 - b*cos(x)^2 - a)*sqrt(-a*cos 
(x)^2 + b*cos(x)^2 + a)) + 3*log(abs(-sqrt(-a + b)*cos(x) + sqrt(-a*cos(x) 
^2 + b*cos(x)^2 + a)))/((a^2 - 2*a*b + b^2)*sqrt(-a + b)))/sgn(sin(x))
 
3.1.55.9 Mupad [F(-1)]

Timed out. \[ \int \frac {\cot ^2(x)}{\left (a+b \cot ^2(x)\right )^{5/2}} \, dx=\int \frac {{\mathrm {cot}\left (x\right )}^2}{{\left (b\,{\mathrm {cot}\left (x\right )}^2+a\right )}^{5/2}} \,d x \]

input
int(cot(x)^2/(a + b*cot(x)^2)^(5/2),x)
 
output
int(cot(x)^2/(a + b*cot(x)^2)^(5/2), x)